Clean up benchmarks #465
Clean up benchmarks #465
Conversation
benches/distributions.rs
Outdated
| let mut accum = 0u32; | ||
| for _ in 0..::RAND_BENCH_N { | ||
| let x: char = distr.sample(&mut rng); | ||
| accum = accum.wrapping_add(x as u32); |
There was a problem hiding this comment.
if this is the only difference, I think you could just do the same for booleans, though doesn't really matter
| for _ in 0..::RAND_BENCH_N { | ||
| black_box(rng.gen_bool(p)); | ||
| accum ^= rng.gen_bool(p); | ||
| p += 0.0001; |
There was a problem hiding this comment.
Adjusting p seems sensible, but it adds an extra op, making this incomparable with misc_gen_bool_const. Was the old method not okay?
There was a problem hiding this comment.
Although LLVM didn't move the range set-up code out of the inner loop, I see no reason why it couldn't. Changing p makes sure it can not.
Funny thing: using black_box before the for loop is slower than incrementing p each round.
|
Updated to fold the |
| for _ in 0..::RAND_BENCH_N { | ||
| black_box(rng.sample(d)); | ||
| let d = rand::distributions::Bernoulli::new(p); |
There was a problem hiding this comment.
In this case I think the point was to check what affect using an unknown p has over using a constant, not to adjust the distribution each loop and do a single call for each instance.
There was a problem hiding this comment.
That is also possible to benchmark, but then we should expect the benchmark to give the same result as the const version. Does not seem like something all that useful to measure to me.
This includes the change as discussed for the
gen_boolenBernoullibenchmarks.Also it turns out almost all our uses of
black_boxwere unnecessary, it is enough to return a variable or some accumulator at the end of the benchmark run.Results with
cargo benchcmp --threshold 2:misc_gen_boolandmisc_bernoullibenchmarks are clearly different (in part because of usingStdRng).boolorcharusing an accumulator gives more realistic results.Hc128RngandStdRngproduce results somewhere between 1250 MB/s and 1800 MB/s, without recompilation. Still don't know what causes the variance.black_boxfrom thegen_bytesbenchmarks, it improves performance a bit more than I think is plausible, but I didn't want to figure out the assembly for those...